Whole-genome sequencing surveillance and machine learning for healthcare outbreak detection and investigation: A systematic review and summary.

Background: Whole-genome sequencing (WGS) has traditionally been used in infection prevention to confirm or refute the presence of an outbreak after it has occurred. Due to decreasing costs of WGS, an increasing number of institutions have been utilizing WGS-based surveillance. Additionally, machine learning or statistical modeling to supplement infection prevention practice have also been used. We systematically reviewed the use of WGS surveillance and machine learning to detect and investigate outbreaks in healthcare settings. Methods: We performed a PubMed search using separate terms for WGS surveillance and/or machine-learning technologies for infection prevention through March 15, 2021. Results: Of 767 studies returned using the WGS search terms, 42 articles were included for review. Only 2 studies (4.8%) were performed in real time, and 39 (92.9%) studied only 1 pathogen. Nearly all studies (n = 41, 97.6%) found genetic relatedness between some isolates collected. Across all studies, 525 outbreaks were detected among 2,837 related isolates (average, 5.4 isolates per outbreak). Also, 35 studies (83.3%) only utilized geotemporal clustering to identify outbreak transmission routes. Of 21 studies identified using the machine-learning search terms, 4 were included for review. In each study, machine learning aided outbreak investigations by complementing methods to gather epidemiologic data and automating identification of transmission pathways. Conclusions: WGS surveillance is an emerging method that can enhance outbreak detection. Machine learning has the potential to identify novel routes of pathogen transmission. Broader incorporation of WGS surveillance into infection prevention practice has the potential to transform the detection and control of healthcare outbreaks.

Whole-Genome Sequencing Surveillance and Machine Learning of the Electronic Health Record for Enhanced Healthcare Outbreak Detection.

BACKGROUND: Most hospitals use traditional infection prevention (IP) methods for outbreak detection. We developed the Enhanced Detection System for Healthcare-Associated Transmission (EDS-HAT), which combines whole-genome sequencing (WGS) surveillance and machine learning (ML) of the electronic health record (EHR) to identify undetected outbreaks and the responsible transmission routes, respectively. METHODS: We performed WGS surveillance of healthcare-associated bacterial pathogens from November 2016 to November 2018. EHR ML was used to identify the transmission routes for WGS-detected outbreaks, which were investigated by an IP expert. Potential infections prevented were estimated and compared with traditional IP practice during the same period. RESULTS: Of 3165 isolates, there were 2752 unique patient isolates in 99 clusters involving 297 (10.8%) patient isolates identified by WGS; clusters ranged from 2-14 patients. At least 1 transmission route was detected for 65.7% of clusters. During the same time, traditional IP investigation prompted WGS for 15 suspected outbreaks involving 133 patients, for which transmission events were identified for 5 (3.8%). If EDS-HAT had been running in real time, 25-63 transmissions could have been prevented. EDS-HAT was found to be cost-saving and more effective than traditional IP practice, with overall savings of $192 408-$692 532. CONCLUSIONS: EDS-HAT detected multiple outbreaks not identified using traditional IP methods, correctly identified the transmission routes for most outbreaks, and would save the hospital substantial costs. Traditional IP practice misidentified outbreaks for which transmission did not occur. WGS surveillance combined with EHR ML has the potential to save costs and enhance patient safety.

Supplementing Existing Societal Risk Models for Surgical Aortic Valve Replacement With Machine Learning for Improved Prediction.

Background This study evaluated the role of supplementing Society of Thoracic Surgeons (STS) risk models for surgical aortic valve replacement with machine learning (ML). Methods and Results Adults undergoing isolated surgical aortic valve replacement in the STS National Database between 2007 and 2017 were included. ML models for operative mortality and major morbidity were previously developed using extreme gradient boosting. Concordance and discordance in predicted risk between ML and STS models were defined using equal-size tertile-based thresholds of risk. Calibration metrics and discriminatory capability were compared between concordant and discordant patients. A total of 243 142 patients were included. Nearly all calibration metrics were improved in cases of concordance. Similarly, concordance indices improved substantially in cases of concordance for all models with the exception of deep sternal wound infection. The greatest improvements in concordant versus discordant cases were in renal failure: ML model (concordance index, 0.660 [95% CI, 0.632-0.687] discordant versus 0.808 [95% CI, 0.794-0.822] concordant) and STS model (concordance index, 0.573 [95% CI, 0.549-0.576] discordant versus 0.797 [95% CI, 0.782-0.811] concordant) (each P<0.001). Excluding deep sternal wound infection, the concordance indices ranged from 0.549 to 0.660 for discordant cases and 0.674 to 0.808 for concordant cases. Conclusions Supplementing ML models with existing STS models for surgical aortic valve replacement may have an important role in risk prediction and should be explored further. In particular, for the roughly 25% to 50% of patients demonstrating discordance in estimated risk between ML and STS, there appears to be a substantial decline in predictive performance suggesting vulnerability of the existing models in these patient subsets.

Using machine learning to improve risk prediction in durable left ventricular assist devices.

Risk models have historically displayed only moderate predictive performance in estimating mortality risk in left ventricular assist device therapy. This study evaluated whether machine learning can improve risk prediction for left ventricular assist devices. Primary durable left ventricular assist devices reported in the Interagency Registry for Mechanically Assisted Circulatory Support between March 1, 2006 and December 31, 2016 were included. The study cohort was randomly divided 3:1 into training and testing sets. Logistic regression and machine learning models (extreme gradient boosting) were created in the training set for 90-day and 1-year mortality and their performance was evaluated after bootstrapping with 1000 replications in the testing set. Differences in model performance were also evaluated in cases of concordance versus discordance in predicted risk between logistic regression and extreme gradient boosting as defined by equal size patient tertiles. A total of 16,120 patients were included. Calibration metrics were comparable between logistic regression and extreme gradient boosting. C-index was improved with extreme gradient boosting (90-day: 0.707 [0.683-0.730] versus 0.740 [0.717-0.762] and 1-year: 0.691 [0.673-0.710] versus 0.714 [0.695-0.734]; each p<0.001). Net reclassification index analysis similarly demonstrated an improvement of 48.8% and 36.9% for 90-day and 1-year mortality, respectively, with extreme gradient boosting (each p<0.001). Concordance in predicted risk between logistic regression and extreme gradient boosting resulted in substantially improved c-index for both logistic regression and extreme gradient boosting (90-day logistic regression 0.536 versus 0.752, 1-year logistic regression 0.555 versus 0.726, 90-day extreme gradient boosting 0.623 versus 0.772, 1-year extreme gradient boosting 0.613 versus 0.742, each p<0.001). These results demonstrate that machine learning can improve risk model performance for durable left ventricular assist devices, both independently and as an adjunct to logistic regression.

Performance of a Machine Learning Algorithm in Predicting Outcomes of Aortic Valve Replacement.

BACKGROUND: This study evaluated the performance of a machine learning (ML) algorithm in predicting outcomes of surgical aortic valve replacement (SAVR). METHODS: Adult patients undergoing isolated SAVR in The Society of Thoracic Surgeons (STS) National Database between 2007 and 2017 (n = 243,142) were randomly split 4:1 into training and validation sets. Outcomes that were evaluated were those for which STS models exist. The ML algorithm extreme gradient boosting (XGBoost) was used. Model calibration was measured by the ratio of observed to expected risk, calibration-in-the-large, and slope of calibration curve, and model discrimination was measured by the c-index. RESULTS: XGBoost demonstrated excellent calibration, with an average observed-to-expected ratio of 0.985, calibration-in-the-large of -0.017, and slope of calibration curve of 0.944. The c-index of XGBoost was significantly improved compared with STS models for 5 of 7 outcomes: operative mortality (77.1% [95% confidence interval {CI}, 75.8% to 78.4%] vs 76.2% [95% CI, 75.0% to 77.6%]; P = .007), prolonged ventilation (73.9% [95% CI, 73.1% to 74.6%] vs 72.6% [95% CI, 71.9% to 73.4%]; P < .001], acute renal failure (77.6% [95% CI, 76.3% to 78.7%] vs 73.7% [95% CI, 72.2% to 75.0%]; P < .001), reoperation (63.7% [95% CI, 62.7% to 64.8%] vs 62.6% [95% CI, 61.5% to 63.7%]; P = .01), and the composite of mortality or major morbidity (70.3% [95% CI, 69.6% to 70.9%] vs 69.0% [95% CI, 68.3% to 69.7%]; P < .001). For 2 outcomes the c-index was comparable: stroke (68.4% [95% CI, 66.6% to 70.3%] vs 67.6% [95% CI, 65.7% to 69.5%]; P .08) and deep sternal wound infection (59.9% [95% CI, 53.6% to 66.2%] vs 64.1% [95% CI, 57.5% to 70.1%]; P = .82). CONCLUSIONS: The ML algorithm XGBoost demonstrated excellent calibration and modest improvements in discriminatory ability compared with existing STS models in this study of isolated SAVR.

Outbreak of Pseudomonas aeruginosa Infections from a Contaminated Gastroscope Detected by Whole Genome Sequencing Surveillance.

BACKGROUND: Traditional methods of outbreak investigations utilize reactive whole genome sequencing (WGS) to confirm or refute the outbreak. We have implemented WGS surveillance and a machine learning (ML) algorithm for the electronic health record (EHR) to retrospectively detect previously unidentified outbreaks and to determine the responsible transmission routes. METHODS: We performed WGS surveillance to identify and characterize clusters of genetically-related Pseudomonas aeruginosa infections during a 24-month period. ML of the EHR was used to identify potential transmission routes. A manual review of the EHR was performed by an infection preventionist to determine the most likely route and results were compared to the ML algorithm. RESULTS: We identified a cluster of 6 genetically related P. aeruginosa cases that occurred during a 7-month period. The ML algorithm identified gastroscopy as a potential transmission route for 4 of the 6 patients. Manual EHR review confirmed gastroscopy as the most likely route for 5 patients. This transmission route was confirmed by identification of a genetically-related P. aeruginosa incidentally cultured from a gastroscope used on 4of the 5 patients. Three infections, 2 of which were blood stream infections, could have been prevented if the ML algorithm had been running in real-time. CONCLUSIONS: WGS surveillance combined with a ML algorithm of the EHR identified a previously undetected outbreak of gastroscope-associated P. aeruginosa infections. These results underscore the value of WGS surveillance and ML of the EHR for enhancing outbreak detection in hospitals and preventing serious infections.

Predictive Utility of a Machine Learning Algorithm in Estimating Mortality Risk in Cardiac Surgery.

BACKGROUND: This study evaluated the predictive utility of a machine learning algorithm in estimating operative mortality risk in cardiac surgery. METHODS: Index adult cardiac operations performed between 2011 and 2017 at a single institution were included. The primary outcome was operative mortality. Extreme gradient boosting (XGBoost) models were developed and evaluated using 10-fold cross-validation with 1000-replication bootstrapping. Model performance was assessed using multiple measures including precision, recall, calibration plots, area under the receiver-operating characteristic curve (C-index), accuracy, and F1 score. RESULTS: A total of 11,190 patients were included (7048 isolated coronary artery bypass grafting [CABG], 2507 isolated valves, and 1635 CABG plus valves). The Society of Thoracic Surgeons Predicted Risk of Mortality (STS PROM) was 3.2% ± 5.0%. Actual operative mortality was 2.8%. There was moderate correlation (r = 0.652) in predicted risk between XGBoost and STS PROM for the overall cohort and weak correlation (r = 0.473) in predicted risk between the models specifically in patients with operative mortality. XGBoost demonstrated improvements in all measures of model performance when compared with the STS PROM in the validation cohorts: mean average precision (0.221 XGBoost vs 0.180 STS PROM), C-index (0.808 XGBoost vs 0.795 STS PROM), calibration (mean observed-to-expected mortality: XGBoost 0.993 vs 0.956 STS PROM), accuracy (1%-3% improvement across discriminatory thresholds of 3%-10% risk), and F1 score (0.281 XGBoost vs 0.230 STS PROM). CONCLUSIONS: Machine learning algorithms such as XGBoost have promise in predictive analytics in cardiac surgery. The modest improvements in model performance demonstrated in the current study warrant further validation in larger cohorts of patients.

Automated data mining of the electronic health record for investigation of healthcare-associated outbreaks.

BACKGROUND: Identifying routes of transmission among hospitalized patients during a healthcare-associated outbreak can be tedious, particularly among patients with complex hospital stays and multiple exposures. Data mining of the electronic health record (EHR) has the potential to rapidly identify common exposures among patients suspected of being part of an outbreak. METHODS: We retrospectively analyzed 9 hospital outbreaks that occurred during 2011-2016 and that had previously been characterized both according to transmission route and by molecular characterization of the bacterial isolates. We determined (1) the ability of data mining of the EHR to identify the correct route of transmission, (2) how early the correct route was identified during the timeline of the outbreak, and (3) how many cases in the outbreaks could have been prevented had the system been running in real time. RESULTS: Correct routes were identified for all outbreaks at the second patient, except for one outbreak involving >1 transmission route that was detected at the eighth patient. Up to 40 or 34 infections (78% or 66% of possible preventable infections, respectively) could have been prevented if data mining had been implemented in real time, assuming the initiation of an effective intervention within 7 or 14 days of identification of the transmission route, respectively. CONCLUSIONS: Data mining of the EHR was accurate for identifying routes of transmission among patients who were part of the outbreak. Prospective validation of this approach using routine whole-genome sequencing and data mining of the EHR for both outbreak detection and route attribution is ongoing.

Statistical outbreak detection by joining medical records and pathogen similarity.

We present a statistical inference model for the detection and characterization of outbreaks of hospital associated infection. The approach combines patient exposures, determined from electronic medical records, and pathogen similarity, determined by whole-genome sequencing, to simultaneously identify probable outbreaks and their root-causes. We show how our model can be used to target isolates for whole-genome sequencing, improving outbreak detection and characterization even without comprehensive sequencing. Additionally, we demonstrate how to learn model parameters from reference data of known outbreaks. We demonstrate model performance using semi-synthetic experiments.

Development of the Pseudomonas aeruginosa mushroom morphology and cavity formation by iron-starvation: a mathematical modeling study.

We present a mathematical model of mushroom-like architecture and cavity formation in Pseudomonas aeruginosa biofilms. We demonstrate that a proposed disparity in internal friction between the stalk and cap extracellular polymeric substances (EPS) leads to spatial variation in volumetric expansion sufficient to produce the mushroom morphology. The capability of diffusible signals to induce the formation of a fluid-filled cavity within the cap is then investigated. We assume that conversion of bacteria to the planktonic state within the cap occurs in response to the accumulation or depletion of some signal molecule. We (a) show that neither simple nutrient starvation nor signal production by one or more subpopulations of bacteria is sufficient to trigger localized cavity formation. We then (b) demonstrate various hypothetical scenarios that could result in localized cavity formation. Finally, we (c) model iron availability as a detachment signal and show simulation results demonstrating cavity formation by iron starvation. We conclude that iron availability is a plausible mechanism by which fluid-filled cavities form in the cap region of mushroom-like structures.